Towards an Integrative Information Society: Studies on Individuality in Speech and Sign

The flow of information within modern information society has increased rapidly over the last decade. The major part of this information flow relies on the individual’s abilities to handle text or speech input. For the majority of us it presents no problems, but there are some individuals who would benefit from other means of conveying information, e.g. signed information flow. During the last decades the new results from various disciplines have all suggested towards the common background and processing for sign and speech and this was one of the key issues that I wanted to investigate further in this thesis. The basis of this thesis is firmly within speech research and that is why I wanted to design analogous test batteries for widely used speech perception tests for signers – to find out whether the results for signers would be the same as in speakers’ perception tests. One of the key findings within biology – and more precisely its effects on speech and communication research – is the mirror neuron system. That finding has enabled us to form new theories about evolution of communication, and it all seems to converge on the hypothesis that all communication has a common core within humans. In this thesis speech and sign are discussed as equal and analogical counterparts of communication and all research methods used in speech are modified for sign. Both speech and sign are thus investigated using similar test batteries. Furthermore, both production and perception of speech and sign are studied separately. An additional framework for studying production is given by gesture research using cry sounds. Results of cry sound research are then compared to results from children acquiring sign language. These results show that individuality manifests itself from very early on in human development. Articulation in adults, both in speech and sign, is studied from two perspectives: normal production and re-learning production when the apparatus has been changed. Normal production is studied both in speech and sign and the effects of changed articulation are studied with regards to speech. Both these studies are done by using carrier sentences. Furthermore, sign production is studied giving the informants possibility for spontaneous speech. The production data from the signing informants is also used as the basis for input in the sign synthesis stimuli used in sign perception test battery. Speech and sign perception were studied using the informants’ answers to questions using forced choice in identification and discrimination tasks. These answers were then compared across language modalities. Three different informant groups participated in the sign perception tests: native signers, sign language interpreters and Finnish adults with no knowledge of any signed language. This gave a chance to investigate which of the characteristics found in the results were due to the language per se and which were due to the changes in modality itself. As the analogous test batteries yielded similar results over different informant groups, some common threads of results could be observed. Starting from very early on in acquiring speech and sign the results were highly individual. However, the results were the same within one individual when the same test was repeated. This individuality of results represented along same patterns across different language modalities and - in some occasions - across language groups. As both modalities yield similar answers to analogous study questions, this has lead us to providing methods for basic input for sign language applications, i.e. signing avatars. This has also given us answers to questions on precision of the animation and intelligibility for the users – what are the parameters that govern intelligibility of synthesised speech or sign and how precise must the animation or synthetic speech be in order for it to be intelligible. The results also give additional support to the well-known fact that intelligibility in fact is not the same as naturalness. In some cases, as shown within the sign perception test battery design, naturalness decreases intelligibility. This also has to be taken into consideration when designing applications. All in all, results from each of the test batteries, be they for signers or speakers, yield strikingly similar patterns, which would indicate yet further support for the common core for all human communication. Thus, we can modify and deepen the phonetic framework models for human communication based on the knowledge obtained from the results of the test batteries within this thesis.Siirretty Doriast

Puhujan identiteetin ja äänensävyjen välittyminen kirjoitustulkkeessa

Artikkelissa käsitellään puhujan identiteetin ja äänensävyjen välittymistä kirjoitustulkkauksessa ja niitä keinoja, joita tulkilla on käytössään, sekä erilaisten tilanteiden vaikutusta eri keinojen käyttöön. Aihetta käsitellään yhtäältä kirjallisuuden, toisaalta käytännön tulkkaustilanteissa havaittujen seikkojen pohjalta. Tulkkaustilanteisiin liittyvät esimerkit kertovat erilaisista tulkkaustilanteista ja -ratkaisuista.</p

Coarticulation in sign and speech

Proceedings of the NODALIDA 2009 workshop Multimodal Communication &mdash; from Human Behaviour to Computational Models. Editors: Costanza Navarretta, Patrizia Paggio, Jens Allwood, Elisabeth Alsén and Yasuhiro Katagiri. NEALT Proceedings Series, Vol. 6 (2009), 21-24. © 2009 The editors and contributors. Published by Northern European Association for Language Technology (NEALT) http://omilia.uio.no/nealt . Electronically published at Tartu University Library (Estonia) http://hdl.handle.net/10062/9208

Accessibility of TV media for the dual-sensory impaired in Finland and in the UK

Peer reviewe

Role of Different Spectral Attributes in Vowel Categorization: the Case of Udmurt

Proceedings of the 16th Nordic Conference of Computational Linguistics NODALIDA-2007. Editors: Joakim Nivre, Heiki-Jaan Kaalep, Kadri Muischnek and Mare Koit. University of Tartu, Tartu, 2007. ISBN 978-9985-4-0513-0 (online) ISBN 978-9985-4-0514-7 (CD-ROM) pp. 384-388

PUHEEN JA VIITTOMIEN SUHDE: FONEETTINEN TUTKIMUS

Puheen gesturaalisesta näkökulmasta puheen ja viittomisen välinen yhteys on kiistaton.Käsien eleet ovat aivojemme kannalta verrattavissa ääntöelimistön liikkeisiin.Puheen havaitseminen ei perustu itse puhesignaaliin, vaan sen synnyttävienartikulaatioliikkeiden ymmärtämiseen. Viime kädessä puheen ja viittomien havaitseminenja tuottaminen tulisi nähdä yhtenä ja samana aivojen tapahtumasarjana,josta ne eivät ole erotettavissa motorisen ja sensorisen ulottuvuutensa perusteella.Viittomakieli perustuu vastaavanlaiseen merkkijärjestelmään kuin puhutun kielenfoneemit. Näin ollen puheen tutkimusta varten kehitettyjen foneettisten havaintokokeidenvoidaan ajatella soveltuvan myös viittomakielen tutkimukseen. Esimerkkikokeessammeselvitettiin, miten viittomakieliset, ei-viittomakieliset ja viittomakielentulkithavaitsevat viitottuja ärsykkeitä. Alustavat tulokset osoittavat, että viittomaärsykkeillätehdyssä havaintokokeessa eri koehenkilöryhmien välillä on havaittavissasamankaltaisia eroavaisuuksia kuin vokaaleilla tehdyissä havaintokokeissa.Näin ollen voidaan päätellä, että havaitseminen tapahtuu samoin periaattein sekäpuhe- että viittomakielisillä.Avainsanat: Puhe, motorinen teoria, viittomatKey words: Speech, motor theory ofspeech, sig

Accessibility of TV media for the dual-sensory impaired in Finland and in the UK

One of the most used media is television. Almost all of us have a television set at home. Television is commonly thought also to be accessible, and for most of us, it seems to be so. However, there are people who cannot enjoy televised programmes without special measures. These measures include subtitling, audio description and sign language interpretation. All these are mentioned in the in-force EU Audiovisual Media Services Directive.In this article we look at how these are broadcast in different countries, what equipment is needed for receiving them, and how accessible the programme information (EPG) and menus to choose these accessibility options are, as they most often are not automatic, but have to be turned on in the accessibility options menu or the TV set or digital terrestrial receiver box (digibox). This article focuses on accessibility with special interest on the dual-sensory impaired perspective over four days comparing two countries: the UK where all three accessibility features mentioned above are well-established and Finland, where accessibility features concentrate mainly on subtitling.</p

Procedia Social and Behavioral Sciences

In this article we introduce a systematic, holistic way of experiencing music performances through touch. This method called social-haptic communication, which consists of various touch messages called haptices is frequently used in interpreting for sensory impaired people, but can also be applied to various other disability groups, especially in connection to therapeutic approaches. Haptices are mostly used in interpreting visual events, but here we expand the scope into acoustic events, more specifically into interpretation of music. In this article we use a concert as an example of the versatility of haptices in sharing a music experience. The various techniques used in training to use haptices with music are also explained in further detail.<br /

Feeling vibrations from a hearing and dual-sensory impaired perspective

Working with hearing and dual-sensory impaired clients presents challenges for music therapy professionals. Feeling and experiencing the vibrations produced by music help to understand what the concept of music is. Music and vibroacoustic therapies have overlapping effects. Music is for listening while vibroacoustic therapy is mainly similar to physiotherapy. Where vibroacoustic (VA) facilities are not available, some flexible methods could be adapted. One can use a music centre with two separate, moveable speakers positioned on a wooden floor to enhance music vibrations. In conjunction inflatable balloons can be manipulated by the clients. Balloons can be held towards the sound source and used to test the variations of the intensity and dynamics of the vibrations in the room. The choice of music styles plays an important role to amplify the vibrations and introduce music to the clients. Clients’ feedback was very positive indicating they were able to feel some musical tones from low, middle and high levels. These methods appeared to enhance musical vibrations and give some therapeutic experiences, i.e. relaxation and awareness of musical tones. This approach has been tested over 25 years and the individual feedback supports notions on how tones are felt through the body.</p